Common Illumination between Real and Computer Generated Scenes

The ability to merge a real video image (RVI) with a computer-generated image (CGI) enhances the usefulness of both. To go beyond "cut and paste" and chroma-keying, and merge the two images successfully, one must solve the problems of common viewing parameters, common visibility and common illumination. The result can be dubbed Computer Augmented Reality (CAR). We present in this paper techniques for approximating the common global illumination for RVIs and CGIs, assuming some elements of the scene geometry of the real world and common viewing parameters are known. Since the real image is a projection of the exact solution for the global illumination in the real world (done by nature), we approximate the global illumination of the merged image by making the RVI part of the solution to the common global illumination computation. The objects in the real scene are replaced by few boxes covering them; the image intensity of the RVI is used as the initial surface radiosity of the visible part of the boxes; the surface reflectance of the boxes is approximated by subtracting an estimate of the illuminant intensity based on the concept of ambient light; finally global illumination using a classic radiosity computation is used to render the surface of the CGIs with respect to their new environment and for calculating the amount of image intensity correction needed for surfaces of the real image. An example animation testing these techniques has been produced. Most of the geometric problems have been solved in a relatively ad hoc manner. The viewing parameters were extracted by interactive matching of the synthetic scene with the RVIs. The visibility is determined by the relative position of the "blocks" .representing the real objects and the computer generated objects, and a moving computer generated light has been inserted. The results of the merging are encouraging, and would be effective for many applications. Resume La possibilite d' integrer une image video reelle (lVR) et une image de synthese (IS) augmente considerablement I'utilite des deux. Pour pouvoir aller au-dela de "l' ecran bleu" et de la simple composition, on doit resoudre les problemes de parametres de camera, de visibilite, et surtout d' illumination commune. L'ensemble de ces techniques est appele realite augmentee par images de synthe'se (RAIS). Nous presentons dans cet article des techniques pour approximer I'illumination globale pour des IVR et des IS, en supposant quelques elements connus pour la geometrie de la scene et des parametres de camera. L'illumination globale entre les vrais objets est bien sur deja calculee, et nous utilisons ces resulats dans !'image finale . Les objets de la vraie scene sont remplaces a fin de calcul d'illumination et de visibilite par des "boites" qui les recouvrent. L'intensite obtenue de l'IVR est utili see pour une premiere approximation de la radiosite des parties visibles de la boite d' un objet. La reflectance de la surface est estimee par approximation de la lumiere ambiante, et finalement le calcul normal de radiosite est fait avec les objets de synthese, l' approximation des vrais objets et les sources lumineuses synthese et vraies si elles sont connues. Une animation testant ces concepts a ete realisee. Nous avons inclus les mouvements de la camera et le deplacement d'un objet reel. Une source lumineuse de synthese se deplace egalement pendant l' animation. Les resultats sont encourageant, et sont utilisables pour des applications pratiques.